The quality of a service from the perspective of a user consuming that service functions as an important measure of the service's end-to-end performance. This proves particularly true for services like video streaming (e.g., IPTV) that rely on a delivery network for transport, because poor delivery network performance greatly affects the video streaming quality perceived by the user. Armed with information about user-perceived quality (also known as quality of user experience), a network operator can better plan for future network development, monitor user satisfaction, make adjustments to network processing and/or storage resources, make adjustments to video stream encoding, and the like.
One common approach to measuring user-perceived quality of a video stream entails directly surveying users regarding their subjective opinion about that quality. For example, users may be asked to rate the quality of a video stream on a predefined scale (e.g., from 1 to 5) for different video stream bitrates. The average of these subjective ratings for a given bitrate represents the Mean Opinion Score (MOS) for that bitrate. Of course, subjective approaches like this require large human resources and are quite time-consuming, which prevents the approaches from being used dynamically for real-time applications like quality monitoring.
Other approaches can be implemented dynamically because they estimate the user-perceived quality of a video stream based on objective criteria, rather than based on the subjective opinions of users. One objective approach, for example, directly evaluates decoded video frames of a stream for certain characteristics that are modeled as being associated with user-perceived quality. Such characteristics may be the degree of motion present in the decoded video frames (e.g., jerkiness), the degree of detail present in the decoded video frames (e.g., blockiness or blurriness), and other visual artifacts present in the decoded video frames. While this objective approach proves advantageous for a number of reasons, one serious drawback to the approach remains that it cannot be used if the video stream payload is encrypted.
Another objective approach addresses this problem of encryption by evaluating parameters contained within one or more packet-layer headers that encapsulate the video stream payload, rather than evaluating the video stream payload itself. These packet-layer parameters may describe, for instance, the codec, format, frame rate, and bitrate of the video stream and may be contained within any number of packet layers (e.g., from the IP layer through the RTP layer and the MPEG2-TS layer).
Notably, however, none of the packet-layer parameters directly describe the underlying video content of the video stream, e.g., in terms of whether the video content relates to fast moving content (such as a hockey game) or to slow moving content (such as a newscast). Because of this, known packet-layer objective approaches fail to adequately account for the affect that the underlying video content of a video stream has on the user-perceived quality of that stream. Indeed, such known approaches may estimate that different video streams have the same user-perceived quality for a given bitrate, even though the streams in fact have very different qualities (e.g., on the order of several MOS values) because they relate to very different video content. Such inaccurate estimation of user-perceived quality ultimately degrades that quality, since any network adjustments made based on that estimation may be less effective in improving end-to-end performance.